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SYSTEM FOR TWO DIMENSIONAL AND THREE DIMENSIONAL 
IMAGING OF TUBULAR STRUCTURES IN THE HUMAN BODY 

Field of the Invention 

5 The present invention generally relates to anatomical modeling of the human 

body with a computer, and more particularly to a computerized system for analyzing 
tubular structures of the human body such as the digestive tract of a living person for 
detecting colorectal polyps. 

10 Background of the Invention 

In industrialized nations, colorectal cancer is the second leading cause of 

deaths from malignancy. In the United States, almost 150,000 people are found to 

have colon cancer annually and it causes approximately 60,000 deaths annually. 

Only lung cancer causes more deaths. Colon cancers are preventable because they 
1 5 usually begin as benign polyps which grow slowly for five to ten years before 

becoming cancerous. If these polyps are detected and removed, the risk of 

developing colon cancer is greatly reduced. 

Unfortunately, widespread colorectal screening and preventive efforts are 

hampered by several practical impediments, including limited resources, 
20 methodologic inadequacies, and poor patient acceptance leading to poor compliance. 

In addition, a fecal occult blood test (FOBT) fails to detect the majority of cancers 

and pre cancerous polyps. Since sigmoidoscopy only examines a portion of the 

colon, it also misses many polyps. The accuracy of barium enema varies among 

centers and is therefore not always reliable. 
25 Therefore, there is a need for a new test which can be used to screen for pre 

cancerous colon polyps. Like all screening tests, this new test must be relatively 

inexpensive, minimally invasive, and highly specific. 

A technique using helical computed tomography (CT) to create computer 

simulated intraluminal flights through the colon was proposed as a novel approach 
30 for detecting colorectal neoplasms by Vining DJ, Shifrin RY, Grishaw EK, Liu K, 

Gelfand DW, Virtual colonoscopy (Abst) , Radiology Scientific Prgm 1994; 

193(P):446. This technique was first described by Vining et al. in an earlier abstract 
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by Vining DJ, Gelfand DW, Noninvasive colonoscopy using helical CT scanning, 
3D reconstruction, and virtual reality (Abst) , SGR Scientific Program, 1994. This 
technique, referred to as "virtual colonoscopy", requires a cleansed colon insufflated 
with air, a helical CT scan of approximately 30 seconds, and specialized three- 
5 dimensional (3D) imaging software to extract and display the mucosal surface. The 
resulting endoluminal images generated by the CT scan are displayed to a medical 
practitioner for diagnostic purposes. 

The technique of reformatting 2D cross sections perpendicular to the colon 
midline is also described in U.S. Patent No. 5,458,1 11, issued October 17, 1995 to 
10 Coin. However, direct interpretation of the cross-sectional images is difficult 

because a scan of the colon consists of several hundred axial tomograms. Without 
advanced image manipulation tools, interpretation of the colon's complex three 
dimensional shape from these cross sections alone is very difficult for a medical 
practitioner. 

1 5 One approach to improve accuracy involves production of reformatted 2D 

images at cross sections and orthogonal angles to the colon midline. These 
reformatted 2D images can provide complimentary diagnostic information when 
viewed with corresponding intraluminal 3D images. Exam efficiency can be 
improved with innovative 3D rendering techniques that allow fast, interactive 

20 evaluation on low priced computer hardware. 

Retained fluid in the lumen of a colon is also a commonly encountered 
problem that can obscure lesions. Prone imaging in addition to standard supine 
imaging is often required to visualize obscured colonic segments. Although the 
colorectum is often seen optimally with combined supine and prone views, twice as 

25 much interpretation is required with both acquisitions. 

Furthermore, the current CTC utilizes an interactive combination of axial, 
reformatted 2D and 3D images (from an endoluminal perspective) that are generated 
in real time. Navigating the colorectum within this "Virtual Endoscopy" (VE) 
metaphor is tedious. To alleviate this problem and free the radiologist to concentrate 

30 on diagnosis, a two-pass approach has been adopted. In the first pass, the colon is 
semiautomatically navigated and its midline is recorded. Then in a second pass, the 
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radiologist moves the view point (virtual endoscope tip) along this'midline. This 
second pass takes from several minutes per scan. 

To reduce interpretation time of Computed Tomographic Colography (CTC), 
additional volume rendered displays have been developed which show longer 
5 segments of the colon in formats analogous to views which may be seen at autopsy. 
These include a technique called "Planar Virtual Pathology" (PVP) which uses 
image planes of the two longitudinal transcolonic sections as cut planes within 
isometric volume rendered images. Scenes are rendered with rays passing 
orthogonally through these planes from both sides. Each of the four resulting 

10 images depicts approximately one-half of the inner surface of a 12 cm bowel 
segment. Interpretation is performed by viewing these colon segments at 
approximately 8 cm intervals. In this way, the entire colon can be rapidly examined 
with a minimum of navigational input from a user such as a radiologist. 

Other virtual pathology views have also been developed. These include 

1 5 Cylindrical Virtual Pathology (CVP) and Mercator Virtual Pathology (MVP). CVP 
views are formed by casting rays perpendicular to a straight line which approximates 
a segment of the colon midline. These views are analogous to the result of splitting 
a segment of excised colon and opening it to expose its inner surface. Interpretation 
can be performed interactively or by viewing a series of pre-computed CVP views as 

20 described for PVP above. With MVP, views are generated by casting rays from a 
single viewpoint in all directions. The resulting view is a panoramic image. 
Interpretation of CVP is performed interactively in a manner similar to that of VE. 

In resolving the above problem, volume rendered endoluminal perspective 
views of the colon have been explored. However, available workstations were not 

25 fast enough to do the required rendering interactively. One of the methods of 
addressing this challenge is to render six images at each point along the colon 
midline. These images are aligned with the image coordinates and each has a ninety 
degree field of view. Virtual Endoscopy (VE) views with arbitrary camera 
orientation and view field can be derived from these images in real time using 

30 simple image warp and merge operations. These panoramic scene sequences make it 
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possible to visualize the entire endoluminal surface without further manual 
navigation. 

Additional panoramic projections have been developed. They are volume 
rendered scenes. However, they are quite different from other perspective volume 
5 renderings. In conventional perspective rendering techniques, pixel coordinates map 
the angle (or the tangent thereof) between the cast ray and the central axis of scene. 
The range of angles (field of view) is usually limited to around +/- 45 degrees. The 
present invention produces images where these angles range from +/- 90 degrees in 
the y-axis (latitude) and 0 to 360 degrees in x-axis (longitude). The result is a 

1 0 panoramic endoluminal view similar to a mercator projection map. This is called 
Mercator Virtual Pathology (MVP). Another technique provided by the present 
invention is to map angles of view from 0 to 360 degrees along the x-axis but to 
move the point of view for each row in the image. This forms a cylindrical 
projection. This is called Cylindrical Virtual Pathology (CVP). The advantage of 

1 5 each of these techniques of the present invention is that it shows larger regions of the 
colon in each rendered scene than previous virtual colonographic images. They also 
simplify navigation by eliminating two or three of the degrees of freedom. 

Therefore, there is a need for techniques which provide efficient and accurate 
evaluation of the colon using helical CT data. There is a need for displaying 

20 coronal, sagittal, and axial views of the colon. There is also a need for displaying a 
three dimensional image of the colon. There is a further need for displaying an 
unfolded or open view of the colon. 

Yet there is a need to further reduce interpretation time of the Computed 
Tomographic Colography. There is also a need for an improved display mode which 

25 enables a user to interpret long segments of the colon at one time and to reduce 
interpretation time for Computed Tomographic Colography. 



Summary of the Invention 
The present invention provides a system, method, and article of manufacture 
30 for imaging tubular structures of the human body, such as the digestive tract of a 
living person, with a medical imaging device such as a computed tomography (CT) 
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scanner and a computer workstation. The CT scanner is used to generate cross- 
sectional axial images of a human colon which are then transferred to the computer 
workstation. A colon midline is defined which follows the colon lumen. The 
computer workstation supports colon midline definition by generating and 
5 displaying reformatted cross-sectional images, volume rendered scouts, and 

interluminal views. Semi-automatic midline defining tools are also included. After 
the midline is defined, a montage of images is displayed for diagnostic purposes. 
The images include axial sections, transluminal cross section, and intraluminal 
volume rendered images. 

1 0 Semiautomatic methods are used to delineate the three dimensional shape of 

the colon by identifying its approximate midline. This is supported by the display of 
coronal, sagittal, and axial views as well as an off-axis plane orthogonal to the 
midline. Straightened curved sections along the midline and a rendered view from 
the end of the midline are also displayed. Editing the midline in any of these views 

1 5 causes the other views to update accordingly. After the midline is traced, data are 
extracted so the colon can be examined efficiently. 

A medical practitioner, such as a radiologist, examines the colon by moving 
the view point along the delineated midline. Three orthogonal off-axis cross 
sections, volume rendered extraluminal scouts or the raw axial 2D images, and a 

20 high resolution perspective rendering of the colon's inner surface are all displayed. 
The perspective views can be re-oriented in any direction. A rotatable longitudinal 
sectioning along the colon's midline is also displayed. This view, showing the entire 
length of the colon, enhances both navigation of the path forward or backward along 
the 3D images and interpretation of the 3D images. 

25 Empirical research shows that the simultaneous display of cross-sectional 

and rendered views enhances a diagnostic interpretation more than either cross- 
sectional or intraluminal views alone. Volume rendering is performed by custom 
algorithms that use pre-computation of texture and other techniques to achieve 
interactive performance on moderately priced workstations. These greatly reduce 

30 the number of computations required to generate a ray traced image, which makes 
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volume rendering possible with computational requirements similar to those of 
surface rendering. 

Two views of the three dimensional images are displayed to the observer. 
The first view is a forward intraluminal view which encompasses a view of the colon 
5 from the view point looking away from a terminating location of the colon, such as 
the anal verge. The second view is a backward intraluminal view which 
encompasses a view of the colon from the view point looking toward a terminating 
location of the colon, such as the anal verge. Displaying both views makes it less 
likely that a feature of interest will be obscured due to the topology of the colon. 

10 By pointing the cursor and simultaneously pressing a key, an observer can 

move the view point off the previously defined midline, and all images, including 
2D reformatted images, 3D intraluminal images, and the 2D axial image are updated 
to views corresponding to the designated position. A fiducial mark on each of the 
images identifies the new position. The observer can randomly move the view point 

1 5 by using the pointing device, or return to the nearest point on the predetermined 
midline. 

To denote orientation and location, two extraluminal renderings of the air- 
filled colon are displayed in axial and coronal orientations to assist the observer in 
determining location and orientation. 

20 To assist in the diagnosis, an unfolded or opened view of the colon is also 

displayed to the observer. The opened view of the colon corresponds to a view of 
the entire colon as if it had been physically divided and opened for inspection. 

The present invention also provides an alternative method, system, and 
article of manufacture for reduction in interpretation time of scanned image data 

25 involved in the production of reformatted 2D images at cross sections and 

orthogonal, etc. One of the ways is to display synchronously both prone and supine 
scanned images of the colon. This approach enhances the ability of Computed 
Tomographic Colography (CTC) that combines with advantages of prone and supine 
views without the added interpretation times of reviewing two separate scans. 

30 Simultaneous display of synchronized (anatomically registered) views of the colon 
eliminates the need for two separate readings of the colon and shortens interpretation 
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time. In one embodiment of the present invention, the anatomic levels are indexed 
to match each other and are advanced synchronously as a radiologist interprets the 
data set. Axial, reformatted 2D and 3D images are displayed and simultaneously 
updated for both prone and supine images on the same computer screen. The colon 
5 only heeds to be reviewed once, with the diagnostic benefit of both supine and prone 
scans. In many cases, the two scans can be interpreted nearly as quickly as one scan. 

Accordingly, the present invention is also a computed tomographic 
colography (CTC) system having such improved interpretation tools which provide 
synchronously interactive montage of images. The present system displays several 

10 distinct types of images. These include volume rendered endoluminal perspective 
(virtual colonoscopic views), transcolonic sections, navigational scout images 
showing the entire colon, and "mathematically unraveled" views of the colon. The 
region of the screen where the scouts and small axial, sagittal, and coronal views are 
normally displayed can alternately be used to present a single enlarged axial section. 

15 The present system can also include a virtual pathology software which displays 
rendered images of a large colon segments in order to greatly reduce interpretation 
time. 

Therefore, as opposed to viewing the scan results through tracing along a 
midline of the colon to find a desired position, the alternative method of the present 

20 invention allows a user (such as a radiologist) to freely view a desired position of a 
subject any time without following a scan sequence tracing along the midline of the 
colon. Furthermore, because of the ability of freely viewing a desired position of a 
subject and the ability of displaying dual scanned images of the subject 
simultaneously according to the corresponding indexes established during the dual 

25 scanning, a clearer image of the subject at the desired position is shown by 

comparing the two views (supine and prone) and selectively enlarging the better 
view of the two views. Accordingly, this approach not only significantly reduces the 
interpretation time by cutting down the pre-processing of the scanned data tracing 
along the midline, but also largely increases the quality of the image. 

30 The present invention also provides a viable colon screening tool wherein the 

perspective volume rendering approach (dubbed virtual colonoscopy) is augmented 
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by alternate rendering scheme called "Virtual Pathology (VP)" to facilitate a faster, 
global viewing approach. The present invention provides CTC reading approaches 
which are based on this new metaphor. It will be appreciated that a person skilled in 
the art would recognize that a method of interpreting CTC will be altered as these 
5 new type of rendered images continue to grow in acceptance and improve in speed 
and image quality. 

Other objects, features, and characteristics of the present invention; methods, 
operation, and functions of the related elements of the structure; combination of 
parts; and economies of manufacture will become apparent from the following 
10 detailed description of the preferred embodiments and accompanying drawings, all 
of which form a part of this specification, wherein like reference numerals designate 
corresponding parts in the various figures. 



Brief Description of the Drawings 
1 5 FIG. 1 is a perspective view of the components of a system for imaging a 

human body with a medical imaging device compatible with the present invention. 

FIG. 2 is a block diagram illustrating an exemplary hardware environment 
used to implement a preferred embodiment of the invention. 

FIG. 3 is a flow diagram illustrating a method for imaging a human body 
20 with a medical imaging device compatible with the present invention. 

FIG. 4 is a screen display of a midline tracing tool used for semi-automatic 
tracing of a colon midline compatible with the present invention. 

FIG. 5 is a screen display of cross section and orthogonal views to the colon 
centerline in a patient with a pedunculated polyp compatible with the present 
25 invention. 

FIG. 6 is a screen display of a diagnostic image collage compatible with the 
present invention. 

FIGs. 7a through 7c show the splining of a hypothetical midline by various 
algorithms. 

30 FIG. 8 is a screen display of a parameter adjustment window compatible with 

the present invention. 
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FIG. 9 is a flow diagram illustrating a method for rendering a 3D image 
compatible with the present invention. 

FIG. 10 is a schematic diagram illustrating a ray casting method compatible 
with the present invention. 
5 FIG. 1 1 is a screen display of a diagnostic image collage compatible with the 

present invention illustrating dual scanned images. 

FIG. 12 is a screen display of an enlarged image collage of one of the dual 
scanned images selected by a user compatible with the present invention. 

FIG. 13 is a screen display of an example of a Panoramic Virtual Endoscopy 
10 (PVE) compatible with the present invention. 

FIG. 14 is a screen display of an example of a Cylindrical Virtual Pathology 
(CVP) compatible with the present invention. 

FIG. 1 5 is a screen display of an example of a Mercator Virtual Pathology 
(MVP) compatible with the present invention. 
15 FIG. 16 is a screen display of a type of a Planar Virtual Pathology (PVP) and 

the corresponding tomographs compatible with the present invention. 

FIG. 17 is a screen display of an Axial Planar Virtual Pathology (APVP) 
compatible with the present invention. 

FIG. 18 is a flow diagram illustrating a method for imaging a human body 
20 with a medical imaging device providing synchronous interpretation of supine and 
prone computed tomographic colography. 

Detailed Description of the Preferred Embodiments 
In the following detailed description of the preferred embodiments, reference 
25 is made to the accompanying drawings which form a part hereof, and in which is 
shown by way of illustration specific embodiments in which the invention may be 
practiced. It is to be understood that other embodiments may be utilized and 
structural changes may be made without departing from the scope of the present 
invention. 



30 
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Glossary 

The following terms and phrases are used throughout the application. 

1 . Computed tomographic colonography (CTC). A technique combining 
5 ' helical or volumetric CT imaging of the colon with imaging software to 

produce reformatted 2D and 3D images of the colon. 

2. Reformatted 2D images. Images which are oriented at cross section and two 
orthogonal angles to the colon midline. 

3. 3D intraluminal image. An image which simulates an endoscopic view of 
10 the colon. The outside of the colon is not visualized. 

4. 2D axial image. The original image produced by the helical or volumetric 
CT scanner before any image post-processing is performed. 

5. Straightened colon / continuous orthogonal image. Images which are 
oriented at a cross section in the volume image of the straightened colon 

1 5 which correspond to a convoluted ribbon bisecting the colon along its axis. 

6. 3D extraluminal images. Images which represent as 3D extraluminal 
rendering of the entire colon in axial, sagittal and coronal orientations. 

7. Raw 2D images. Transaxial non-reformatted images generated by the CT 
scanner. 

20 

Detailed Description 

For the purpose of illustrating the best mode of practicing the present 
invention, this description describes the process of analyzing the colon of a living 
human patient. It will be recognized that the present invention can enhance the 

25 assessment of any tubular structure of the human body where intraluminal and 

transluminal views are desirable. By way of example, the present invention has been 
used to evaluate the trachea and bronchial tree, the bowel, and other vascular 
structures such as the aorta. One of ordinary skill in the art will recognize that other 
organs and structures of the human body in addition to those listed above, such as 

30 the heart, stomach, or bladder, may be analyzed with the present invention without 
loss of generality. 
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FIG. 1 shows the components of a preferred medical imaging system that 
may be used with the present invention. The system includes a workstation typically 
having a keyboard 10 by which a user may input data into the system, a computer 
chassis 12 which holds electrical processing components and a memory 18. The 
5 computer 12 also contains a removable storage device 20 by which removable media 
22, typically floppy disks, magnetic tapes, or removable fixed disks, may be used to 
provide information to the computer 12. The computer 12 could also be coupled to 
other I/O devices, including a local area network (LAN) or wide area network 
(WAN) via interface cable 19. The system further contains a screen display 14 by 

10 which information 24 is displayed to the user, and a pointing device 16, typically a 
mouse. The computer components are logically connected to each other via internal 
system bus within the computer, which is further connected to a helical or 
volumetric CT scanner 26 via an external data bus 28. The present invention 
includes a preprogrammed set of instructions which are stored in the memory 1 8 and 

1 5 processed by one or more processing units within the computer 12. 

FIG. 2 is a block diagram illustrating an exemplary hardware environment 
used to implement the preferred embodiment of the invention. In the exemplary 
hardware and software environment, the computer 12 may include, inter alia, a 
processor 30, memory 18, keyboard 10, screen display 14, as well as fixed and/or 

20 removable data storage devices and their associated media 20, 22, and 32. Those 
skilled in the art will recognize that any combination of the above components, or 
any number of different components, peripherals, and other devices, may be used 
with the computer 12. 

The present invention is generally implemented using a compiler program, 

25 and a linker program, each of which executes under the control of an operating 
system 34, such as OS/2, Windows, AIX, UNIX, DOS, etc. For purposes of 
simplification, the compiler program and linker program will be collectively referred 
to herein as a compiler 36. Those skilled in the art will recognize that the compiler 
36 can conform to many programming language conventions without departing from 

30 the scope of the present invention. A graphical user interface (GUI) 38 executes 
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under the control the operating system 34 to provide text and graphics information 
on the screen display 14. 

Those skilled in the art will recognize many modifications may be made to 
this configuration without departing from the scope of the present invention. 
5 Computed tomography colonography (CTC) typically uses a helical 

computed tomography scanner to obtain contiguous axial cross sections of the lower 
abdomen. It will be recognized that other imaging technologies may be used instead 
of helical computed tomography, such as electron beam CT, magnetic resonance 
imaging (MRI), positron emission tomography (PET), or soft x-rays or high- 

1 0 resolution ultrasound. Because of the nature of CT data, typically seventy five to 

five hundred images are produced for a single series. The present invention provides 
an efficient way to handle the large number of images, reduce the amount of time 
required for diagnosis, and improve the reliability of the interpretation. 

A preferred method of performing a CTC examination involves four sub- 

15 processes: 1) preparation of the colon, 2) scanning the colon, 3) identification of the 
colon midline, and 4) support of diagnostic interpretation by a radiologist. Dividing 
analysis steps into separate sub-processes is a significant departure from the 
paradigm of most medical imaging workstations. Current workstations, like most 
personal computer and workstation packages, typically are based upon a "toolbox" 

20 metaphor. The operator of these systems, like a craftsman in a work shop or an artist 
in a studio, has access to all of the tools and raw materials required to construct a 
finished product, which is in this case a medical diagnosis. The choice of tools and 
the order of their use are controlled by the operator. Given enough time, a skilled 
artist can construct a wide variety of finished products. However, the artist is not 

25 optimally efficient and would typically find it difficult to produce several identical 
products. 

In contrast, the present invention provides for the users of CTC systems to 
operate in a manner similar to assembly line workers. Each receives intermediate 
results from a previous subsystem, performs a limited number of well-defined tasks, 
30 has access to the tools appropriate for the performance of these tasks, and passes the 
partially finished product on to the next stage in a well-defined state of 
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completeness. Just as assembly lines are more efficient than craftsman, systems 
based upon the present invention's "pipeline" metaphor are often more efficient than 
those based upon the "toolbox" metaphor. Additionally, the present invention 
insures that the same processing steps are performed on each image data set, making 
5 it much more appropriate for scientific applications. 

FIG. 3 is a flow diagram illustrating a method for imaging a human body 
with a medical imaging device compatible with the present invention. At block 40, 
the medical imaging device 26 is used to generate an image data set of the structure 
within the human body. At block 42, the image data set is used to assist in defining 

10 a midline following the structure lumen. The midline may be manually defined, or 
semiautomatic tools may assist the user in creating the midline. At block 44, the 
computer 12 uses the image data set to automatically generate two dimensional 
reformatted images of the structure centered on the structure midline. At block 46, 
the computer 12 automatically generates a three dimensional intraluminal image of 

15 the structure. At block 48, the two dimensional reformatted images and the three 
dimensional intraluminal image of the structure are dynamically simultaneously 
displayed to the user on the display device 14. 

Processing 

20 As described above, the preferred method of performing a CTC examination 

can be divided into four sub-processes: 1) preparation of the colon, 2) scanning the 
colon, 3) identification of the colon midline, and 4) support of diagnostic 
interpretation by a radiologist. The first two sub-processes are generally known 
within the prior art, but are briefly described below because they are significant to 

25 understanding the subsequent steps. The remaining two sub-processes are described 
in detail below. 



Process 1 : Preparation of the Colon 

Initially, the present invention provides for colon preparation equivalent to 
30 that of traditional colonoscopy including fasting and the oral administration of 

laxatives. The patient's colon is insufflated with air. It will be recognized that other 
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colon preparations may be used with the present invention, such as insufflating the 
colon with a gas other than air, filling the colon with a liquid contrast medium, using 
an oral stool marker in either the prepped or unprepped colon, or eliminating 
insufflation or filling of the colon altogether. It will also be recognized that a 
5 modification of the colon preparation procedure should be accompanied by 

corresponding modifications to the image analysis steps described below, because 
the modification will likely change the appearance of the colon's inner surface in the 
medical images. 

10 Process 2: Scanning the Colon 

Scanning is preferably performed on a fast helical CT scanner. As noted 
above, other medical imaging technologies may be substituted for a fast helical CT 
scanner without loss of generality. This technology requires a volumetric data set 
that covers the entire structure. Although a variety of scanning protocols are used, a 

1 5 typical collimation of five millimeters (mm) and a typical pitch of 1 -2 is used. If a 
table speed of five mm per second is used, ten centimeters can be scanned in a 20 
second breath hold. Since the colon spans a thirty to fifty centimeter region of the 
lower abdomen, three or four breath holds are required. Results produced by the 
present invention will improve as the technological improvement of CT scanners 

20 provides for machines which are capable of scanning the entire colon in a single 
breath hold. A reconstruction interval of one to three mm is often used. 

Process 3 : Identification of the Colon Midline 

Following the scanning step described above, the reconstructed images are 

25 transferred to a computer workstation for identification of the colon. The midline, or 
midline, of the colon is delineated through the use of special purpose three 
dimensional tracing software. This software contains a full complement of manual 
and semiautomatic tools. Since the purpose of this subsystem is only to map the 
colon midline, optimal quality images are not necessary. Therefore, the scan is 

30 subsampled to the resolution of the workstation used as the display device. A 

preferred voxel volume of approximately eight cubic mm and a preferred contrast 
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resolution of eight data bits is used with one embodiment of the present invention, 
although it will be recognized that the voxel volume and contract resolution may 
each be substantially increased or decreased without loss of generality. By using the 
preferred voxel volume and contrast resolution, the amount of data which must be 
5 processed is typically reduced by approximately a factor of sixteen. 

The computer workstation preferably displays images in a montage of views, 
including axial, coronal, and sagittal cross sections and corresponding rendered 
scout views, as shown in FIG. 4. Fiducial marks shown within each of the displayed 
views provide three dimensional localization. As the three dimensional course of 

10 the colon is delineated, the colon midline is preferably superimposed over the scout 
views and intersections with cross sections are displayed. Extension of the 
delineated midline or movement of the cursor in any image causes corresponding 
changes in the other images. 

FIG. 4 illustrates a workstation display showing a preferred software 

1 5 interface of the present invention. The first three images on the upper row are axial 
50, coronal 52, and sagittal 54 cross sections, respectively. Corresponding rendered 
scouts axial 56, coronal 58, and sagittal 60, are shown just below each of these 
views. The image 62 in the upper right portion of the display is an enlarged off axis 
cross section centered over the end of a delineated midline. Below this image is a 

20 low resolution rendered view 64 looking further into the colon. The elongated 
pictures at the bottom of the display 66, 68 are images of the straightened colon 
showing the entire length of the delineated colon. At the bottom of the display, a 
graphical user interface (GUI) control panel 70 allows a user to adjust the window 
and level settings. A pop-up window, not shown in FIG. 4, is used to access a tool 

25 kit of automatic tracing tools and similar options. Other windows, such as a 

parameter adjustment window, shown in FIG. 8, are used to adjust other settings and 
parameters within the system. 

The preferred reduced scouts are depth shaded views of all intra-abdominal 
air. The rendering uses thresholding to identify the object to be rendered and very 

30 fast projection along cardinal directions. Depth buffers corresponding to each of 
these projections are also generated. These depth buffers are loaded with the 
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average of the front surface coordinate and the first back surface coordinate. The 
buffers are used to determine three dimensional coordinates when a point is selected 
on one of the scouts. This feature simplifies pointing at the center of the colon. 

Once the tracing of the colon midline has begun, four additional images are 
5 preferably displayed, as shown in FIG. 4. The first two images include 1) an off- 
axis cross section that is centered on the midline's endpoint and orthogonal to its 
final segment, and 2) a depth shaded intraluminal perspective volume rendered view 
from and aligned with the midline's endpoint. 

The remaining two images are an innovative improvement of the "curved 

10 section" displays in well-known prior art medical imaging workstations. In 
conventional curved section displays, one axis is kept parallel to one cardinal 
direction in the scanned volume while the other tracks the specifying curve in the 
other two directions. The orthogonal pair of neo-curved sections included in the 
colon tracing subsystem of the present invention adapts to the extreme tortuosity of 

1 5 the colon and displays the identified midline as a straight line bisecting the displays. 
Each column of displayed imagery is preferably taken from along a line orthogonal 
to the colon midline in the scanned volume. 



Manual Delineation of the Colon Midline 

20 The midline can be identified manually in several ways. In a first method, 

coordinates can be identified by selecting a position in any of the scout views, 
shown in FIG. 4. Since the present invention automatically finds a point near the 
center of the colon, this is a quick and easy' way to trace the colon. When the course 
of the colon is very tortuous, or the desired segment is obscured by other anatomy, 

25 the point identified may not correspond to the desired depth. In these cases, points 
can be selected in the reformatted 2D images. 

A second effective method for extending the midline is to enter new points 
along the segment of the neo-curved sectional images which are beyond the current 
endpoint. Since the two views are orthogonal, alternately selecting new points in 

30 these two views can be used to extend the midline while keeping it centered in the 
colon. 
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It will be noted that the off-axis cross section is centered on the midline. 
Thus, the section will typically show a cross section of the colon that is centered on 
the image and is nearly circular. When points are entered in this image, the 
midline's endpoint is adjusted rather than extended. The image is therefore an 
5 effective tool for keeping the midline centered on the colon lumen. 

Ten images are produced to facilitate centerline definition: 2D axial and 
reformatted coronal and sagittal images as shown in FIG. 5, 3D extraluminal 
renderings in axial, coronal and sagittal views as shown in FIG. 6, a mathematically 
straightened colon in coronal and sagittal orientations as shown in FIGs. 4 and 6; 

1 0 and a 2D cross section and a 3D interluminal view as shown in FIGs. 4 and 6. The 
observer begins at the anal verge and advances through the colon until reaching the 
cecum using any of the ten displayed images. The orientation of the midline can be 
ascertained using the 2D cross section or 3D intraluminal views, and can be adjusted 
by the observer on these images. The observer may remove any preceding step as 

1 5 needed by using the keyboard and can then resume tracing from the last correct 
point. 

The mathematically straightened views of the colon greatly facilitate midline 
delineation. Occasionally the path of the colon, especially in the sigmoid, may be 
difficult to define. In this case, the observer can use the 2D axial and reformatted 
20 coronal and sagittal images to determine the path of the sigmoid colon without 
interrupting the trace. The pointing device on the workstation, typically a mouse, 
allows the observer to scroll through 2D images in order to determine the path of the 
colon. Permanent trace points are placed using the pointing device. 

25 Automatic Methods for Colon Midline Delineation 

Several automated approaches to colon midline delineation are known within 
the prior art. A first approach is the "Marching Cubes" algorithm which finds the 
shortest path between two identified points which, as used in the subject matter of 
this invention, stays within the lumen of the colon. Since the start and end points 

30 must be specified, this can not truly be considered totally automatic. In addition, the 
algorithm is also computationally intensive. Further, the path found by the 
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algorithm is the shortest and thus does not pass along the center of "the colon. 
Instead, it will pass along the inner surface of each curve, which frequently results in 
obstructed views accompanied by disorienting abrupt changes in direction. 

A second approach uses K-T transforms to find principal eigen vectors which 
5 track the course of the colon. This algorithm can only be applied where the radius of 
curvature of the colon midline is greater than twice the diameter of the colon. 
Virtually all colons fail to meet this criterion in at least one segment. Therefore, this 
algorithm is typically unreliable in the subject matter of the present invention. 

A third approach would be to morphologically thin a binary volume image of 

10 the colon lumen into a single line and then vectorize the longest resulting path by a 
method similar to algorithms developed for analysis of remotely senses watersheds. 
While this approach is theoretically possible, there are no known prior-art 
implementations which are successful at tracing the colon. In addition, the 
algorithm would most likely be computationally intensive and may not resolve 

1 5 difficulties with highly convoluted segments. 

In addition to the problems with the three automatic midline delineation 
approaches discussed above, none would be able to adapt to discontinuous shapes 
that are typically present in scans which are acquired in multiple breath holds or 
when bowel distention is lost. To adapt to these conditions, each of the above 

20 techniques must accept manual supervision and thus become semiautomatic. For 
any semiautomatic approach to be viable in clinical situations, it typically must be 
completely integrated with a manual system and must be able to perform the tracing 
faster than an operator of a totally manual system. These criteria are quite limiting 
for current workstation technology. Some or all of the above-mentioned approaches 

25 may be acceptable for limited research applications and may be found in future 
systems based upon higher performance computers. 



30 



Two Approaches to Midline Delineation 

The present invention incorporates semiautomatic tools to facilitate midline 
delineation. These are designed to be faster than a supervising operator and to 
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exploit features of the system which are included for manual operation. Each of 
these approaches finds an approximation of the colon midline. 

A first approach extends the midline in the direction of its final segment a 
fixed fraction of the distance to the point where this line would pass out of the colon 
5 lumen.' This point is known a priori because it is acquired by reading the central 
point of the depth shaded intraluminal view which is always present. An off-axis 
cross section is extracted at this new endpoint as described above. A two- 
dimensional region growing algorithm is used to find the center of the lumen on this 
cross section. The midline endpoint is iteratively adjusted until a satisfactory point 
10 is found. Then the midline is again extended in the direction of the last segment and 
the process is repeated. If a point is reached where the adjustment fails to converge, 
or the operator objects to direction of progress, processing is returned to manual 
control. 

A second approach to semiautomatic tracing makes more direct use of the 
1 5 depth shaded intraluminal rendering. The centroid of the visible region of the colon 
lumen is calculated by applying a special case of Green's Theorem to the surface 
represented in the rendered image: If Vis the volume bounded by a closed regular 
surface S, and if u(x,y,z) and v(x,y,z) are scalar functions having continuous second 
partial derivatives, then 

\\\ v (uV 2 v-v V- u)dV = | J, N • (uVv - vVu)dS 

20 

where N is the out normal to the surface S which bounds V. 

The use of Green's Theorem makes this calculation very simple and is 
preferably embedded in the rendering software itself. This technique therefore 
requires little additional computation and is not iterative. 

25 

Midline Interpolation 

The traced colon midline is resampled to uniform length intervals. Various 
methods have been implemented and tested with the present invention. FIG. 7a 
illustrates a midline using linear splining, which results in abrupt changes in 
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direction at each tie point. Cubic and parabolic splines were tried which yielded 
only approximately uniform sampling intervals and were often more erratic than was 
desirable, as shown in FIG. 7b. Second, a modified iterative parabolic spline and 
circular arc spline were used, which yielded uniformly spaced samples but also 
5 produced curves which were erratic. The problem with splines is that in passing 
through the data points, the splines are not smooth. Excessive curvature does not 
create excessive deviation of the line from a desired midline, but it does change the 
direction of the line sufficient to create an undesirable rocking of the perspective and 
off-axis views which are disorienting to the viewer and could potentially cause some 

1 0 small polyps to be missed. 

The present invention therefore preferably uses a circular arc and tangent 
interpolation method. As illustrated by FIG. 7c, an arc which is tangential to the 
current midline end segment and passes through the next point is determined. If the 
next point is behind the current endpoint, then a p radian arc and a connecting 

1 5 tangent are used. If the arc is between p/2 and p, a p/2 arc and a tangent line are 
used. If in any case the radius of the arc is less than 2 cm, the arc is sampled at p/1 6 
radian increments. Otherwise, the sampling is fixed at 1 or 2 mm intervals along the 
colon. This method is shown by FIG. 4. An example of the excessive curvature 
generated by polynomial splines is shown in the center drawing. The same points 

20 are connected by the preferred arc and tangent spline in the lower drawing. 



Process 4: Support of Diagnostic Interpretation 

Upon completion of midline identification, the CTC scan is typically ready 
for diagnostic interpretation by a radiologist. This interpretation is supported by 

25 another application specific image manipulation tool. The radiologist preferably 
explores a colon by interacting with an intuitive user interface to navigate along the 
course of the colon. The manipulation tool presents a synchronous dynamic collage 
of reformatted 2D and rendered images. These images include navigational scouts, 
reformatted 2D cross sections, a 3D intraluminal image, and a straightened colon 

30 image which presents the entire length of the colon. 
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Interactive Navigation Tools 

Two of the rendered views of intra-abdominal air, used for navigation in the 
earlier program described above, are also used to assist navigation. However, it is 
not necessary for the system to use depth maps to determine three dimensional 
5 coordinates. Instead, a point along the midline which is nearest to a projection of the 
selected position is determined, as shown in FIG. 4. This point is selected for the 
new view point and the corresponding reformatted 2D and 3D intraluminal images 
are displayed. 

Positioning along the midline can also be controlled by selecting points in 

10 the reformatted 2D images themselves. As described above in FIG. 4, two of these 
cross sections are aligned with the colon midline. Selection of points away from the 
longitudinal center of these images causes the view point to move at an equivalent 
distance along the midline. In this way, the plane of the other cross section can be 
manipulated to traverse features seen in these images. This same method of 

15 selecting new view points can be used to rapidly scan a region of the colon. When 
points are iteratively selected at the same position on the screen, the view point 
progresses along the colon midline. The speed of this progress is determined by the 
distance between the selected point and the center of the view. 

The position of the midline can also be controlled by selecting points along 

20 the straightened colon image described above. Generally, the length is too long to 
be displayed in its entirety on the workstation's screen. Instead, a segment is 
displayed at full resolution while the entire strip is displayed at a fractional scale. 
The segment to be displayed and the active view point can be selected using the 
scaled down section. Thus, the scaled down section functions in an analogous way 

25 to a horizontal "elevator" control common on graphical user interfaces. The plane of 
this cross section in the volume image of the straightened colon corresponds to a 
convoluted ribbon which bisects the colon along its axis. The orientation of this cut 
plane relative to the axis of the colon can be altered. This is done by selecting 
position in the off-axis cross section which is orthogonal to the midline. When the 

30 orientation is altered, a line is displayed in this section along the intersection of the 
two sections. This line is also ruled so that radial sizes can be measured. 
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The present invention does not require that the view point be limited to the 
midline. Images can be rendered from any position by selecting a view point on the 
2D reformatted images. In an embodiment of the present invention, by pointing the 
cursor and simultaneously pressing the "Shift" key on the keyboard, an observer can 
5 "jump" off the midline and all images, including 2D reformatted images, 3D 

intraluminal images, and the 2D axial image are updated to views corresponding to 
the designated position. A fiducial mark on all images identifies the new position. 
The observer can advance both forward and backward along the new path, or using 
the pointing device, return to the nearest point on the predetermined midline. 

10 

3D Intraluminal Image Display 

A 3D intraluminal image view is rendered that roughly corresponds to the 
video images from colonoscopy. Several prior art investigations use only these 
rendered views, and as such this technique is known as "virtual colonoscopy". As 

15 described above, a variety of rendering methods have been examined for use with 
the present invention which support interactive reorientation. For these methods, the 
view direction is controlled by selection of points in the rendered image. This 
causes the scene to be re-rendered with the selected feature at the center of the view. 
Additionally, the view direction is represented and can be controlled in the 

20 navigational scouts. A bold fiducial mark such as a dot marks the current view point 
in these projections. A line segment extending from this point in the direction of the 
center of the rendered view is also shown. By manipulating the orientation of this 
indicator in the rendered scout, the radiologist can control the view direction with 
clear external reference points. 

25 The display software preferably displays six 2D images: a reformatted 2D 

image, two orthogonal images, a view of the entire straightened colon, an enlarged 
image of one section of the straightened colon, and the raw axial 2D image. Two 
views of the three dimensional intraluminal images are displayed to the medical 
practitioner. The first is a forward view, and the second is a "rear-view" image 

30 looking backwards. Displaying both views makes it less likely that a feature of 
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interest will be obscured due to the topology of the colon. Two 3D extraluminal 
renderings of the colon are displayed in axial and coronal orientations. 

Several interactive features are available, including window and level 
settings, orientation of the 3D camera angle, and zoom adjustment of the 3D camera. 
5 The observer can choose which area of the colon to inspect by choosing a point on 
the extraluminal renderings of the colon or on the two straightened images of the 
colon. A measurement in millimeters from the anal verge is also displayed. Another 
feature allows the observer to inspect the entire colon wall surface. Using the 
reformatted 2D image, the observer can choose a slice of the colon wall to display as 

1 0 the straightened image. By sequentially rotating the orientation of the slice in either 
a clockwise or counterclockwise direction, the entire colon wall can be evaluated. 

An observer typically begins viewing in the anal verge and advances along 
the trace using the workstation keyboard. Images can be displayed in both forward 
and reverse order. Colon polyps and cancers can be detected on both 2D and 3D 

1 5 images. Unlike 3D images, 2D images allow assessment of the colon wall/soft 
tissue interface so that the depth of the colon cancer invasion or other extraluminal 
disease process can be assessed. 

As images are displayed, a bold fiducial mark is placed in the corresponding 
location on the extraluminal images. In this way, the observer knows which areas of 

20 the colon have been inspected. If an abnormality is detected, its location can be 
determined using both the extraluminal images as well as the measure report in 
millimeters from the anal verge. 

Data Rendering 

25 The present invention reduces the amount of time spent examining a patent's 

image data set by utilizing a 3D rendering technique that facilitates fast interactive 
evaluation of the image data set. In the known prior art, two types of data rendering 
are typically used to display 3D images views of the colon. The first type is surface 
rendering and the second type is volume rendering. 

30 Surface rendering relies on a pre-processing step, known as segmentation, to 

identify the inner or mucosal surface of the colon and represent it as a mesh of 
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triangles. A rendered view of the colon can then be generated very quickly by 
projecting each of these triangles onto a plane representing the display. 

Volume rendering works with the 3D array of densities which comprise a CT 
scan volume. Each element or voxel in the array is projected onto a plane to 
5 generate a two dimensional rendered display image. If the entire volume array is 
used, it is possible to depict CT density as translucence in the rendered scene. 

The process typically involved with volume rendering may require the 
evaluation of over one hundred times as much data as surface rendering, so volume 
rendering is typically much slower than surface rendering. For example, to generate 
1 0 an average rendered scene, approximately 20 computations must be performed on 
each volume element. Given that a 300 slice CT scan may contain over 75 million 
voxels, rendering such a volume on a computer capable of performing 100 million 
useful operations per second would require approximately 1 5 seconds. 

Although the volume covered by a CTC scan is approximately 40 liters, the 
1 5 portion of this volume which is visible in an endoluminal view is typically less than 
400 ml. The rendering technique embodied in the present invention uses this fact to 
limit the amount of processing by only evaluating relevant data within the volume 
array. 

FIG. 9 illustrates the preferred rendering method of the present invention. 

20 The rendering process of the present invention starts at block 80 by obtaining a view 
point within the volume array which is used as a virtual camera location. At block 
82 a view direction is obtained which is used as the orientation direction of the 
virtual camera. The view point and view direction may be derived from along the 
colon midline, the centroid of the prior image rendering, or may be manually 

25 specified by the user. For each display point or pixel in the rendered view of the 
image, the rendering process determines the direction for a ray from the view point 
though the pixel at block 84. The ray is then cast along this direction through the 
volume array. At block 86, the ray is searched for an opaque point. The search is 
performed by sampling the volume array at uniform intervals along the ray. The 

30 interval is selected to coincide with the sample interval in the scan, allowing for the 
use of bilinear interpolation, instead of trilinear interpolation, which is typically used 
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in volume rendering but requires nearly twice as many computer operations. At 
block 88, the exact distance from the view point to the opaque point on the ray is 
calculated by linear interpolation of the last two point values of the ray determined at 
block 86. 

5 ' Block 88 results in the determination of a vector from the virtual camera to a 

point on the rendered surface. The centroid of the space visible in the rendered 
scene is preferably determined by applying Green's Theorem over the surface which 
is specified by the sum of all vectors determined at block 90. Rendered scenes 
which reflect the angle between the surface and the cast ray are preferred over 

1 0 simple depth shading. It is also desirable for the scene to reflect the density gradient 
along the cast ray. Estimating the density gradient is performed at block 92, where a 
density is preferably extracted from the volume array by trilinear interpolation to a 
point at a fixed distance beyond the opaque point on the array determined at block 
88. The value of the fixed distance may be varied according to a "burrow depth" 

1 5 parameter, shown in FIG. 8. This enables an observer to manipulate the relative 
weighting of the two display properties, shading and density, described above. The 
actual brightness of a corresponding pixel in the rendered image is determined at 
block 94. The brightness of the pixel is determined as a linear combination of the 
density from block 92 and the distance from the view point to the opaque point on 

20 the ray from block 88. 

In the preferred embodiment of the present invention, the steps shown in 
FIG. 9 are not all performed sequentially. Blocks 80 and 82 are performed 
selectively, then blocks 84 and 86 are performed for all pixels in the rendered scene. 
The processing loop for the earlier portion of this process is typically the most 

25 computationally intensive, but can be made more efficient by isolating it from the 
rest of the process steps. The process loop is performed at successfully higher 
resolutions and over progressively greater depth ranges. The start point for each ray 
in one iteration is determined from the result of the prior iteration, as shown 
schematically in FIG. 10. 

30 Because the rendering of a 3D image stops when the racy tracing vector 

encounters the internal surface of the colon, a large amount of data which is not 
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visible to the observer is eliminated, which greatly reduces the number of 
computations required to generate the ray traced image. This results in a volume 
rendering process with computational requirements similar to that of a surface 
rendering process. It will be recognized that the rendering technique of the present 
5 invention accommodates any view point along any view direction, and that a 
detected surface along the ray may represent structures other than the internal 
surface of the colon. 

Opening the Colon 

1 0 The present invention assists a medical practitioner with the diagnostic 

interpretation of medical images by displaying an unfolded or opened view of the 
colon. The opened view of the colon corresponds to a view of the entire colon as if 
it had been physically divided and opened for inspection. There are at least four 
methods for opening the colon. 

1 5 The first method for opening the colon divides the colon in two along the 

planes of each of the longitudinal cross sections. This division creates a first half 
and a second half of the colon. An isometric volume rendering is then performed for 
two views of the colon, a view of the first half and a view of the second half, looking 
through the planes of each of the longitudinal cross sections. This process typically 

20 works best in segments where the colon is relatively straight, such as the ascending 
colon, descending colon, and possibly the transverse colon. 

The second method for opening the colon divides the entire scan volume in 
two along a curved surface which passes through the colon midline and is parallel to 
one of the cardinal directions of the scan. As in the first opening method described 

25 above, an isometric volume rendering is then performed for the two views of the 
colon looking through the plane created by the division step. 

A third method for opening the colon mathematically straightens the colon 
prior to sectioning, as in the first opening method described above. This 
straightening will typically distort the colon. However, the distortion will typically 

30 be less than is often seen in perspective views. 
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A fourth method for opening the colon involves distorting the colon such that 
rays projected from the straightened colon midline through the colon surface are 
represented as if they were parallel. As in the third opening method described 
above, this will further distort surface features of the colon, but may not necessarily 
5 adversely affect the readability of the colon rendering. 



Synchronous Interpretation of Supine and Prone CTC 

As with any screening examination, false positive results will slightly 
increase the overall cost and risk associated with CTC. Conversely, false negative 

10 results have the much more serious consequences of failing to detect a potentially 
life-threatening condition. The missed polyps are often in colon segments which 
were either liquid-filled or collapsed. 

Dual scanning is an effective solution to eliminating the false results. 
Basically, a subject, in additional to being scanned in a supine position, is 

1 5 repositioned on a scanner table in a prone position for a prone scanning. The images 
which result from each scan are reconstructed, archived, and transferred to an off- 
line interpretation workstations. 

The addition of the prone scan has at least several advantages. One is that 
dependent surfaces which were covered by liquid stool in the supine scan are now 

20 exposed in the prone scan so that false results are substantially eliminated. Further, 
the insufflation gas redistributes so that colon segments which are collapsed on one 
exam will be adequately distended on the other. The prone scan contributes to 
diagnosis even when the colon is adequately dry and distended. It is often not 
possible to distinguish residual solid stools from colon polyps with a single scan (i.e. 

25 the supine scan or the prone scan). In dual scans, however, as a patient is moved 
into the prone position, the residual solid stools are transferred from one surface of 
the colon to the other. Since polyps are not free to move, this repositioning of the 
subject in the prone scan makes solid stools' identification unambiguous. An 
additional advantage of the prone scan is that the location of breaks between breath 

30 held acquisitions are in different places. This may provide uninterrupted views of 
locations which appear broken on the other scan. 
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However, one disadvantage of the dual scanning is that the interpretation 
time of the dual scans doubles because two separate individual interpretations will 
have to be made by a user. The user views and interprets each scan individually by 
tracing along a midline at each time. Furthermore, as stated above, it is very useful 
5 to compare the two sets (e.g. prone and supine) of images. Making these 

comparisons is very difficult with most image interpretation systems due to the fact 
that the scanned images are individually traced. The present invention uses a dual 
scan interpretation tool that establishes corresponding relationship between two 
scans at each particular position and synchronously view the dual scans at a desired 
10 position requested by a user during the interpretation. Accordingly, only one 

interpretation of the two simultaneously displayed images is needed to be made by 
the user. 

As shown in FIG. 1 1, a display screen of dual scanned images is shown. In 
this mode, corresponding axial images 120, 122 from the supine scan and the prone 

15 scan, respectively, are illustrated at the upper portion of the screen display. As 

shown, the descending colon 124 is nearly filled with liquid stool on the supine scan 
(A) but empty and well distended on the prone scan (B). Three transcolonic 
sectional views 128a, 128b, 128c from the supine scan (A) and one endoluminal, 
perspective, volume rendered view 128d appear just below the axial image 20. The 

20 corresponding images 130a, 130b, 130c, and 130d from the prone scan are displayed 
at the bottom of FIG. 1 1 . The scans can be either interpreted independently or 
simultaneously. Synchronous or simultaneous interpretation of the scanned images 
makes it possible to read both supine and prone scans in almost the same amount of 
time as would be required to carefully interpret either scan individually. Since this 

25 amounts to cutting the interpretation time in half, and a radiologist's interpretation is 
the most expensive single aspect of using CTC technology, substantial cost 
reduction of the system has been achieved. There are now two primary display 
formats produced by the system. One of these modes displays images from either 
the supine or the prone acquisitions. The other mode simultaneously displays 

30 images from corresponding points in both scans. 
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Single Scan Display Mode 

FIG. 12 illustrates a single scan display mode showing an enlarged view of 
either a supine scan or a prone scan shown in FIG. 1 1 . A user can switch the display 
between the dual scan display mode and the single scan display mode as later 
5 explained in FIG. 1 8. In the single scan display mode, the upper three fourths of the 
screen are covered by a single enlarged axial tomogram 126. The lower segment of 
the display contains three transcolonic sectional views 132a, 132b, 132c and one 
intraluminal rendered view 132d as described earlier. 

Navigation through an axial stack is done by simply repositioning the 

10 computer mouse over these views. The lateral position of the cursor determines 
which image is displayed. This is the mode of operation which is preferred by 
radiologists for most of the colon interpretation. Only when a suspected lesion or 
other structure of interest is selected for additional viewing are the derived CTC 
views generated. After these additional views are generated, further navigation is 

1 5 performed by interactively moving the view point or the direction of view. Either of 
these can be manipulated by selection of the corresponding mouse button with the 
cursor located over the desired point in any scene. 

There is a choice of two possible interaction methods. In one interaction 
method, a point of view and rendering orientations are controlled by the middle and 

20 left mouse buttons, respectively. In the other configuration, the user identifies an 
object of interest with the middle mouse button. Then, operations of the left mouse 
button move the view point, but the selected object remains at the center of the view. 
This later method reduces the number of operations required to obtain an 
appropriate view. It will be appreciated that other configurations or interactive 

25 methods, such as defining different mouse buttons or other input devices to achieve 
the same function, can be used within the principles of the present invention. 

Further in FIG. 12, there is also a very narrow, iconic menu bar on the right 
edge of the display (not shown). Views are displayed and other control settings are 
adjusted through the use of this menu. Among the views that can be displayed are 

30 two transcolonic sectional views which bisect the selected intraluminal rendering, an 
additional transcolonic sectional view perpendicular to the look direction, the 



WO 98/32371 PCTYUS98/01504 

30 

rendered endoluminal perspective Virtual Endoscopy (VE) view, arid an additional 
rendered view oriented opposite to the main view. Another one of the Graphic User 
Interface (GUI) buttons on this menu allows the system to return to the dual scan 
displaying mode shown in FIG. 1 1 , which is later explained in FIG. 18. 

5 

Dual Scan Viewing Mode 

In a dual scan displaying mode as shown in FIG. 1 1, the dual scan screen 
layout shows one axial image from corresponding levels in the supine and prone 
scans. These images are roughly one half of life-size. The precise scale depends on 

10 a workstation screen size, a reconstruction circle size, and other parameters. There 
are also two sets of transcolonic sectional views 128a-c (or 130a-c) and two sets of 
rendered view 128d (or 130d). One set 128a-d is derived from the supine scan. The 
other set 130a-d is from the prone scan. The position and orientation of the view 
points used in these images are manipulated in the same manner as used in the single 

15 scan mode. 

The GUI menu bar described in the earlier-mentioned system of a single scan 
displaying mode is also available in this dual scan displaying mode. The button 
which allowed transition into the dual scan displaying mode (FIG. 1 1) is replaced by 
a pair of buttons which allow selection of the single scan displaying mode (FIG. 12) 
20 of either the supine or the prone series. 

Automatic interscan registration 

In one embodiment of the present invention, lists of corresponding image 
numbers are maintained. These lists can be used as look up tables. Each time an 

25 image is displayed from one scan, a table look up is performed to select an image 

from the other scan. These look up tables may be calculated from coordinates along 
a traced midline. Some form of colon midline identification can be used in virtually 
all CTC interpretation systems. One scan protocol preferably used is to scan an 
abdomen in multiple breath holds. The option is to scan with shallow breathing by a 

30 subject, such as a human. This would generally distort the colon and can cause 

polyps to be missed. The preferred protocol may include overlapping scans to avoid 
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the possibility of failing to scan a section of the colon. However, this also 
complicates midline extraction and interscan registration. 

One embodiment of the present invention provides an interbreath overlap 
removal technique which can be used to resolve the distortion or faultline and to 
5 reduce' the number of images needed to be interpreted. However, semi-automated 
midline extraction is only possible with a data set if proceeded by breath gap 
resection. The automatic removal of breath overlaps is performed as follows: by 
evaluating image acquisition times and table positions, the break between breath 
holds can be identified. Then correlations are calculated for images on either side of 

10 the identified gap. e.g.: The first image before the gap is compared to the first image 
after the gap, and the second image before the gap is compared to the second image 
after the gap, and etc. The distance between images which are most similar indicates 
the thickness of the overlap. Then images from each side of the gap and separated 
from each other by this distance are compared. The best correlation in the 

1 5 comparison selects the best place to join the two image sets. All of the images 

between the selected pair are temporarily discarded. The resulting volume image is 
a joining of acquisitions from several breaths which has minimal discontinuities at 
the levels where the sets are joined. Semi-automated midline extraction is then 
performed on this volume image. 

20 After the colon midline has been identified, all locations where the colon 

midline crosses a breath gap are identified. The tracings are modified near these 
locations. This modification results in two separate paths through the region where 
scans overlap. Which path is used is determined by the orientation of the view. 
Specifically, the sign of the z component of the heading vector determines which 

25 path is used. This is done in such a way that the faultline artifact resulting from the 
joining of two separate breath hold acquisitions is only displayed in the remotest 
regions of the rendered view. As this faultline is approached, a point is crossed 
where the entire scene can be rendered within the other scan. At this point, the 
alternate path is chosen. This results in only subtle changes in anatomic shapes and 

30 is often unnoticed by the observer. 



WO 98/32371 PCT/US98/01504 

32 

Further, a traceless interpretation can be used to adjust interscan registration. 
One of the ways in which reductions in interpretation time are achieved is to 
abandon midline extraction. The reason for generating the midline is primarily to 
free the interpreting radiologist from having to concentrate on navigating through 
5 the colon. With the axial cine mode of interpretation, it is no longer necessary to 
restrict concentration to a single point along a predetermined flight path. These 
changes have made significantly accelerated interpretation. It would not be 
necessary to revert to the use of the midline tracing methods. However, this 
alternative method of abandoning the extraction of a colon midline has made 

1 0 interscan registration much more difficult. Generally, it is more efficient to only do 
rough automatic registration prior to dual scan interpretation. Even when no midline 
extraction is done, the identification and repair of breath gap faultline artifacts 
described above may be valuable. This is most useful in improving the quality of 
the rendered Virtual Endoscopy (VE) views. Care has been taken to insure that all 

1 5 of the acquired images remain available for interpretation. However, this is a 

potential problem because navigating to a level in one examination, which is in a 
region of overlapping scans in the other examination, forces the software to choose 
which of the two coincident images in the second examination to display. 

One of the simplest approaches to this problem is to concatenate the 

20 acquisitions without attempting to correct for overlap. This causes the examinations 
to be imprecisely registered. While it does burden a user, such as a radiologist, with 
the necessity of additional navigation, this simplistic approach does not too 
adversely impact readability of the joined examinations. 

25 Virtual Pathology For Interpretation of CTC 

"Virtual Pathology" has significant advantages over virtual endoscopy. The 
term "Virtual Pathology" is used to describe a wide variety of volume rendered 
images. The defining characteristic of these displays is the exposure of large regions 
of the colon for rapid interpretation. By breaking from the restrictions that limit and 

30 colonoscopic interpretation (virtual or real) to a confined path, virtual pathology has 
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made it possible to examine the entire colon without having to navigate each 
tortuous bend and fold. 

There are many different rendering techniques which fit the general 
description of virtual pathology. The conceptually simplest rendering technique is 
5 volume rendering of the opaque torso with movable "cut-planes." 

One virtual pathology is called "Planar Virtual Pathology (PVP)" as shown 
in the left half of FIG. 16. Other images, such as the axial image, transcolonic 
sectional views, and/or endoluminal perspective volume rendered view, can be 
displayed on the same screen with the PVP views. PVP technique uses isometric 

1 0 projection volume rendering through transcolonic cut planes. A line that is tangent 
to the colon midline is selected. Then two perpendicular planes which intersect 
along this line are selected. In one embodiment, six centimeters wide and 12 
centimeters long rectangles which are centered over the tangent lines intersect with 
the colon midline are used as cut plane windows. Volume rendering is performed by 

1 5 casting rays through the transparent (air-filled) fraction of these windows. The four 
resulting images, called Planar Virtual Pathology (PVP), show bisected views of a 
colon segment. In this manner, each portion of the colon surface is displayed twice. 

Axial PVP or called Axial Virtual Pathology (A VP) is a related technique 
which applies the cut plane rendering of PVP with the large axial views. Where the 

20 axial plane intersects the air-filled colon, volume rendering is performed. A pair of 
axial views are displayed in FIG. 17. It shows the same axial cross section as 
viewed from the foot (left) and from the head (right). The rendering of these views 
can be performed very quickly. The rendering is only performed where the axial 
images depict intra-abdominal air. There is a great deal of redundancy from one 

25 level to the next. This makes it possible to generate a scene adjacent to one that has 
already been rendered by casting only about ten percent of the required rays. 
Additionally, all of the rendered rays are parallel to the z-axis. These rays can be 
cast much more quickly than rays with arbitrary orientation. 

FIGs. 13-16 illustrate other types of virtual pathologies which can be used 

30 interpret the CTC. FIG. 1 3 shows an example of a Panoramic Virtual Endoscopy 
(PVE). Generally, in a PVE, six images 134a-l 34f at each point along the colon 
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midline are rendered. These images are aligned with the image coordinates and each 
has a ninety degree field of view. These panoramic images provide the entire 
endo luminal surface without further manual navigation. FIG. 14 illustrates an 
example of a Cylindrical Virtual Pathology (CVP). Generally, CVP views are 
5 formed by casting rays perpendicular to a straight line which approximates a 
segment of the colon midline. The CVP views may be analogous to the result of 
splitting a segment of excised colon and opening it to expose its inner surface. 
Interpretation can be performed interactively or by viewing a series of pre-computed 
CVP views. FIG. 15 demonstrates an example of a Mercator Virtual Pathology 

10 (MVP). Generally, MVP views are generated by casting rays from a single view 
point in all directions, which results a panoramic image. Finally, the left portion of 
FIG. 16 shows a Planar Virtual Pathology. Generally, PVP uses image planes of the 
two longitudinal transcolonic sections as cut planes within isometric volume 
rendered images. Images are rendered with rays passing orthogonally through these 

15 planes from both sides. 

Another virtual pathology is called "Curved Cut-plane Virtual Pathology 
(CCVP)" (not shown). Rendering of the colon surface opaque, as is done in PVP, 
seems to be more appealing than the translucent views of Dual Contrast Barium 
Enema (DCBE) and the simulations thereof. A curved cut-plane is determined 

20 which bisects the colon. This plane is extended several centimeters beyond the 
colon surfaces. Two views are rendered. One shows the abdomen with segments 
cut away to reveal the dependent surfaces. The other shows the segment which was 
excised. 

It will be appreciated to a person skilled in the art to recognize that other 
25 types virtual pathologies can be used to interpret the above discussed Computed 
Tomographic Colography (CTC). 

FIG. 18 shows an operation of imaging a human body with a medical 
imaging device providing synchronous interpretation of supine and prone computed 
tomographic colography and an operation of switching between a dual scan 
30 displaying mode, e.g. as shown in FIG. 1 1 , and a single scan displaying mode, e.g. 
as shown in FIG. 12. The imaging operation starts in box 136. The Computed 
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Tomographic Colography system generates a medical image data set of tubular or 
similar structure in box 138. The generation of image data set can be performed by 
scanning the structure. In boxes 140 and 142, the system determines whether a dual 
scan displaying mode or a single scan displaying mode is requested by a user. If a 

5 dual scan displaying mode is requested from box 1 40, the system interactively 

scrolls through stack of slice pairs in box 144. In box 146, a desired 3 -dimensional 
view and/or cross-sectional views for the supine scan are displayed. Synchronously, 
a desired 3-dimensional view and/or cross-sectional views for the prone scan are 
displayed in box 148. It will be appreciated to skilled in the art that the sequence of 

10 the displays in boxes 146 and 148 is interchangeable and likewise can be 

simultaneous. In box 150, the system is to determine whether the imaging operation 
is done. If yes, the imaging operation ends in box 152. If no, the system determines 
whether a user has requested to switch the display from the dual scan displaying 
mode to the single displaying mode in box 1 54. If no, the imaging operation goes 

15 back to box 144. If a switch request is requested, the system switches to a single 
scan displaying mode on the left side of FIG. 1 8 starting with box 1 56. Likewise, if 
the system initially determines that a user requests a single scan displaying mode in 
box 142, the system goes to execute box 156. In box 156, the system interactively 
scrolls through stack of enlarged tomograms. A desired 3-dimensional view and/or 

20 cross sectional views are displayed in box 158. In box 160, the system generates 
various virtual pathology views as desired. They may include a PVP, MVP, CVP, 
APVP, or Panoramic VE view. In box 162, the system is to determine whether the 
imaging operation is done. If yes, the imaging operation ends in box 152. If no, the 
system determines whether a user has requested to switch the display from the single 

25 scan displaying mode to the dual displaying mode in box 1 64. If no, the imaging 
operation goes back to box 1 56. If a switch request is requested, the system 
switches to a dual scan displaying mode on the right side of FIG. 18 starting with 
box 144. 

It will be appreciated to a person skilled in the art to recognize that multiple 
30 scan displaying modes can be displayed synchronously and switched readily among 
themselves without departing from the principles of the present invention. It will 
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also be appreciated that the imaging and switching operations described in FIG. 1 8 
and the foregoing description of the preferred embodiment of the present invention 
are presented for the purposes of illustration and description. It is not intended to be 
exhaustive or to limit the invention to the precise form disclosed. Many 
5 modifications and variations are possible in light of the above teaching. 

The present invention is to be limited only in accordance with the scope of 
the appended claims, since others skilled in the art may devise other embodiments 
still within the limits of the claims. In addition to perfecting CTC for colon 
screening, the present invention may also be applied to the examination of other 
10 tubular structures within the body. For example, at least the two dimensional 

aspects of the present invention may help evaluate aortic dissection and other CT 
angiography applications. 
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What is claimed is: 

1 . A method for imaging a tubular structure of a human body with a digital 
computer having a processor, a data input device, and a visual display device, the 
5 method comprising the steps of: 

(a) providing to the digital computer an image data set of the structure 
generated by a medical imaging device; 

(b) generating with the processor and the image data set two dimensional 
reformatted images of the structure; 

1 0 (c) generating with the processor a three dimensional image of the 

structure; and 

(d) displaying with the visual display device the two dimensional 
reformatted images and the three dimensional image of the structure. 

15 2. The method of claim 1 , further comprising a step of using the image data set 
to define a structure midline following the structure lumen. 

3. The method of claim 2, wherein the two dimensional reformatted images of 
the structure are centered on the structure midline. 

20 

4. A method for imaging a tubular structure of a human body with a digital 
computer having a processor, a data input device, and a visual display device, the 
method comprising the steps of: 

(a) providing to the digital computer an image data set of the structure 
25 generated by a medical imaging device; 

(b) generating with the processor and the image data set two dimensional 
reformatted images of the structure; 

(c) generating with the processor a two dimensional image of the 
structure; and 

30 (d) displaying with the visual display device the two dimensional 

reformatted images and the two dimensional image of the structure. 
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5. A method for imaging a tubular structure of a human body with a digital 
computer having a processor, a data input device, and a visual display device, the 
method comprising the steps of: 
5 ' (a) providing to the digital computer an image data set of the structure 

generated by a medical imaging device; 

(b) generating with the processor and the image data set two dimensional 
reformatted images of the structure; 

(c) generating with the processor a two dimensional image and a three 
1 0 dimensional image of the structure; and 

(d) displaying with the visual display device the two dimensional 
reformatted images, and the two dimensional image and the three dimensional image 
of the structure. 

15 6. The method of claim 3, wherein the medical imaging device comprises a 
helical scanning device and wherein an axis of rotation of the imaging device is 
substantially perpendicular to a longitudinal axis of the human body. 

7. The method of claim 3, wherein the step of generating the image data set 
20 comprises a reconstruction interval of less than 1 0 millimeters. 

8. The method of claim 3, wherein the step of displaying the three dimensional 
image of the structure comprises displaying the three dimensional image 
simultaneously in an axial view, a sagittal view, and a coronal view. 

25 

9. The method of claim 8, wherein the step of displaying the three dimensional 
image of the structure comprises displaying a distance value representing the 
distance from a first point located at an arbitrary location along the structure to a 
second point located at a terminating location of the structure. 
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1 0. The method of claim 3, wherein the step of displaying the two dimensional 
images comprises displaying the two dimensional images simultaneously in an axial 
view, a sagittal view, a coronal view, and a continuous orthogonal view along a 
straightened version of the structure. 

5 

1 1 . The method of claim 3, wherein the step of defining the structure midline 
comprises manually defining a series of points along the structure lumen. 

12. The method of claim 11, wherein the step of defining the structure midline 

1 0 following the structure lumen comprises generating a virtual spline passing through 
the series of points along the structure lumen, wherein the virtual spline is 
substantially parallel to the structure midline. 

1 3 . The method of claim 1 2, further comprising a step of displaying an 

1 5 intraluminal three dimensional view of the structure by generating a view point and 
a view direction with the structure midline such that a field of view defined by the 
view point and view direction substantially encompasses a view of the structure 
along the structure midline looking away from a terminating location of the 
structure. 

20 

14. The method of claim 12, further comprising the step of displaying an 
intraluminal three dimensional view of the structure by generating a view point and 
a view direction with the virtual spline such that a field of view defined by the view 
point and view direction substantially encompasses a view of the structure along the 

25 structure midline looking toward a terminating location of the structure. 

1 5 . The method of claim 12, further comprising the step of displaying a three 
dimensional view of the structure by using a manually selected view point and a 
view direction which departs from the structure midline. 
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1 6. The method of claim 1 5, wherein the step of displaying an intraluminal three 
dimensional view of the structure by using a manually selected view point and a 
view direction comprises displaying a view of the structure forward and backward 
along a path defined by the manually selected view point and view direction. 

5 

17. The method of claim 16, wherein the step of displaying with the visual 
display device the two dimensional reformatted images and the three dimensional 
image of the structure comprises simultaneously dynamically updating the two 
dimensional reformatted images and three dimensional image forward and backward 

1 0 along the path defined by the manually selected view point and view direction. 



18. The method of claim 3, wherein the step of displaying the three dimensional 
image of the structure comprises the steps of: 

(a) obtaining a view point; 

(b) defining a view direction as a vector pointing toward a centroid of 
one of the two dimensional reformatted images; 

(c) casting a ray through the image data set from the view point along the 
view direction; 

(d) determining a change in density point along the ray; 

(e) projecting the ray past the change of density point; and 

(f) using the ray to display the three dimensional image of the structure. 

1 9. The method of claim 3, wherein the" step of displaying the three dimensional 
image of the structure comprises the steps of: 

(a) defining the structure in two along a plane of longitudinal cross 
section of the structure, defining a first half and a second half; and 

(b) displaying the three dimensional image for a view of either the first or 
second half having a view direction through the plane of a longitudinal cross section 
of the structure. 



30 



WO 98/32371 PCT7US98/01504 

41 

20. A system for imaging a tubular structure of a human body with a digital 
computer having a processor, a data input device, and a visual display device, the 
system comprising: 

(a) data transfer means for providing to the digital computer an image 
5 data set of the structure generated by a medical imaging device; 

(b) reformat means logically coupled to the digital computer for 
generating with the processor two dimensional reformatted images of the structure; 

(c) render means logically coupled to the digital computer for generating 
with the processor a three dimensional image of the structure; and 

1 0 (d) display means logically coupled to the reformat means and render 

means for displaying with the visual display device the two dimensional reformatted 
images and the three dimensional image of the structure. 

2 1 . The system of claim 20, further comprising definition means logically 
1 5 coupled to the digital computer for defining a structure midline following the 

structure lumen. 

22. The system of claim 20, wherein the two dimensional reformatted images of 
the structure are centered on the structure midline. 

20 

23. A system for imaging a tubular structure of a human body with a digital 
computer having a processor, a data input device, and a visual display device, the 
system comprising: 

(a) data transfer means for providing to the digital computer an image 
25 data set of the structure generated by a medical imaging device; 

(b) reformat means logically coupled to the digital computer for 
generating with the processor two dimensional reformatted images of the structure; 

(c) render means logically coupled to the digital computer for generating 
with the processor a two dimensional image of the structure; and 
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(d) display means logically coupled to the reformat means and render 
means for displaying with the visual display device the two dimensional reformatted 
images and the two dimensional image of the structure. 

5 24. A system for imaging a tubular structure of a human body with a digital 
computer having a processor, a data input device, and a visual display device, the 
system comprising: 

(a) data transfer means for providing to the digital computer an image 
data set of the structure generated by a medical imaging device; 
10 (b) reformat means logically coupled to the digital computer for 

generating with the processor two dimensional reformatted images of the structure; 

(c) render means logically coupled to the digital computer for generating 
with the processor a two dimensional image and a three dimensional image of the 
structure; and 

1 5 (d) display means logically coupled to the reformat means and render 

means for displaying with the visual display device the two dimensional reformatted 
images, and the two dimensional and the three dimensional images of the structure. 

25. The system of claim 22, wherein the medical imaging device comprises a 
20 helical scanning device and wherein an axis of rotation of the imaging device is 

substantially perpendicular to the longitudinal axis of the human body. 

26. The system of claim 22, wherein the display means comprises a 
reconstruction interval in the range of between 1 and 10 millimeters. 

25 

27. The system of claim 22, wherein the display means comprises means for 
displaying the three dimensional image simultaneously in an axial view, a sagittal 
view, and a coronal view. 



30 



28. The system of claim 27, wherein the display means comprises means for 
displaying a distance value representing the distance from a first point located at an 
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arbitrary location along the structure to a second point located at a terminating 
location of the structure. 

29. The system of claim 22, wherein the display means comprises means for 

5 displaying the two dimensional reformatted images simultaneously in an axial view, 
a sagittal view, a coronal view, and a continuous orthogonal view along a 
straightened version of the structure. 

30. The system of claim 22, wherein the definition means comprises manually 
10 defining a series of points along the structure lumen. 

3 1 . The system of claim 30, wherein the definition means comprises means for 
generating a virtual spline passing through the series of points along the structure 
lumen, wherein the virtual spline is substantially parallel to the structure midline. 

15 

32. The system of claim 3 1 , wherein the display means comprises means for 
generating a view point and a view direction with the structure midline such that the 
field of view defined by the view point and view direction substantially encompasses 
a view of the structure along the structure midline looking away from a terminating 

20 location of the structure. 

33. The system of claim 3 1 , wherein the display means comprises means for 
generating a view point and a view direction with the virtual spline such that the 
field of view defined by the view point and view direction substantially encompasses 

25 a view of the structure along the structure midline looking toward a terminating 
location of the structure. 

34. The system of claim 3 1 , wherein the display means comprises means for 
displaying a three dimensional view of the structure by using a manually selected 

30 view point and a view direction which departs from the structure midline. 
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35. The system of claim 34, wherein the display means comprises means for 
displaying a view of the structure forward and backward along a path defined by the 
manually selected view point and view direction. 



5 36. The system of claim 22, wherein the display means comprises means for 
simultaneously dynamically updating the two dimensional reformatted images and 
three dimensional image forward and backward along the path defined by the 
manually selected view point and view direction. 



1 0 37. The system of claim 22, wherein the display means comprises means for 
simultaneously dynamically updating the two dimensional reformatted images and 
three dimensional image forward and backward along a path substantially parallel to 
the structure midline. 



38. The system of claim 22, wherein the display means comprises: 

(a) means for obtaining a view point; 

(b) means for defining a view direction as a vector pointing toward a 
centroid of one of the two dimensional reformatted images; 

(c) means for casting a ray through the image data set from the view 
point along the view direction; 

(d) means for determining a change in density point along the ray; 

(e) means for projecting the ray past the change of density point; and 

(f) means for using the ray to display the three dimensional image of the 
structure. 

39. The method of claim 22 wherein the display means comprises: 

(a) means for defining the structure in two along a plane of longitudinal 
cross section of the structure, defining a first half and a second half; and 

(b) means for displaying the three dimensional image for a view of either 
the first or second half having a view direction through the plane of the longitudinal 
cross section of the structure. 



WO 98/32371 



45 



PCT/US98/01504 



40. One or more program storage devices readable by a computer having a 
memory and coupled to a data storage device, each of the program storage devices 
tangibly embodying one or more programs of instructions executable by a computer 

5 to perform method steps for imaging a tubular structure of a human body with a 

digital computer having a processor, a data input device, and a visual display device, 
the method comprising the steps of: 

(a) providing to the digital computer an image data set of the structure 
generated by a medical imaging device; 
1 0 (b) using the image data set to define a structure midline following the 

structure lumen; 

(c) generating with the processor and the image data set two dimensional 
reformatted images of the structure centered on the structure midline; 

(d) generating with the processor a three dimensional image of the 
1 5 structure; and 

(e) displaying with the visual display device the two dimensional 
reformatted images and the three dimensional image of the structure. 

41 . The method of claim 1, wherein the step (a) includes providing at least two 
20 image data sets of the structure, the step (b) includes generating at least two 2- 

dimensional reformatted images of the structure, the step (c) includes generating at 
least two 3 -dimensional image of the structure, and step (d) includes displaying 
synchronously with the visual display device at least the two 2-dimensional 
reformatted images. 

25 

42. The method of claim 1, wherein the step (a) includes providing at least two 
image data sets of the structure, the step (b) includes generating at least two 2- 
dimensional reformatted images of the structure, the step (c) includes generating at 
least two 3 -dimensional images of the structure, and step (d) includes displaying 

30 synchronously with the visual display device at least the two 3-dimensional images 
of the structure. 
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43 . The method of claim 4, wherein the step (a) includes providing at least two 
image data sets of the structure, the step (b) includes generating at least two 2- 
dimensional reformatted images of the structure, the step (c) includes generating at 

5 least two 2-dimensional images of the structure, and step (d) includes displaying 
synchronously with the visual display device at least the two 2-dimensional 
reformatted images. 

44. The method of claim 4, wherein the step (a) includes providing at least two 
1 0 image data sets of the structure, the step (b) includes generating at least two 2- 

dimensional reformatted images of the structure, the step (c) includes generating at 
least two 2-dimensional images of the structure, and step (d) includes displaying 
synchronously with the visual display device at least the two 2-dimensional images 
of the structure. 

15 

45. The method of claim 5, wherein the step (a) includes providing at least two 
image data sets of the structure, the step (b) includes generating at least two 2- 
dimensional reformatted images of the structure, the step (c) includes generating at 
least two 2-dimensional images of the structure and at least two 3 -dimensional 

20 images of the structure, and step (d) includes displaying synchronously with the 
visual display device at least the two 2-dimensional images of the structure. 

46. The method of claim 5, wherein the step (a) includes providing at least two 
image data sets of the structure, the step (b) includes generating at least two 2- 

25 dimensional reformatted images of the structure, the step (c) includes generating at 
least two 2-dimensional images of the structure and at least two 3 -dimensional 
images of the structure, and step (d) includes displaying synchronously with the 
visual display device at least the two 3-dimensional images of the structure. 

30 47. The system of claim 20, wherein the data transfer means provides at least 
two image data sets of the structure, the reformat means generates at least two 2- 
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dimensional reformatted images, the render means generates at least two 3- 
dimensional images of the structure, and the display means displays synchronously 
at least the two 2-dimensional reformatted images. 

5 48. The system of claim 20, wherein the data transfer means provides at least 
two image data sets of the structure, the reformat means generates at least two 2- 
dimensional reformatted images, the render means generates at least two 3- 
dimensional images of the structure, and the display means displays synchronously 
at least the two 3-dimensional images of the structure. 

10 

49. The system of claim 23, wherein the data transfer means provides at least 
two image data sets of the structure, the reformat means generates at least two 2- 
dimensional reformatted images, the render means generates at least two 2- 
dimensional images of the structure, and the display means displays synchronously 

1 5 at least the two 2-dimensional reformatted images. 

50. The system of claim 23, wherein the data transfer means provides at least 
two image data sets of the structure, the reformat means generates at least two 2- 
dimensional reformatted images, the render means generates at least two 2- 

20 dimensional images of the structure, and the display means displays synchronously 
at least the two 2-dimensional images of the structure. 

5 1 . The system of claim 24, wherein the data transfer means provides at least 
two image data sets of the structure, the reformat means generates at least two 2- 

25 dimensional reformatted images, the render means generates at least two 2- 

dimensional images of the structure and at least two 3-dimensional images of the 
structure, and the display means displays synchronously at least the two 2- 
dimensional images of the structure. 

30 52. The system of claim 24, wherein the data transfer means provides at least 
two image data sets of the structure, the reformat means generates at least two 2- 
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dimensional reformatted images, the render means generates at least two 2- 
dimensional images of the structure and at least two 3-dimensional images of the 
structure, and the display means displays synchronously at least the two 3- 
dimensional images of the structure. 

5 

53 . One or more program storage devices readable by a computer having a 
memory and coupled to a data storage device, each of the program storage devices 
tangibly embodying one or more programs of instructions executable by a computer 
to perform method steps for imaging a tubular structure of a human body with a 

1 0 digital computer having a processor, a data input device, and a visual display device, 
the method comprising the steps of: 

(a) providing to the digital computer at least two image data sets of the 
structure generated by a medical imaging device; 

(b) generating with the processor and the image data sets at least two 2- 
15 dimensional reformatted images of the structure; and 

(c) displaying synchronously with the visual display device at least the 
two 2-dimensional reformatted images. 

54. One or more program storage devices readable by a computer having a 

20 memory and coupled to a data storage device, each of the program storage devices 
tangibly embodying one or more programs of instructions executable by a computer 
to perform method steps for imaging a tubular structure of a human body with a 
digital computer having a processor, a data input device, and a visual display device, 
the method comprising the steps of: 

25 (a) providing to the digital computer at least two image data sets of the 

structure generated by a medical imaging device; 

(b) generating with the processor at least two 
3-dimensional images of the structure; and 

(c) displaying synchronously with the visual display device at least the 
30 two 3 -three dimensional images of the structure. 
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55. The method of claim 1, wherein the step of generating of the three 
dimensional image of the structure includes a panoramic Virtual Endoscopy. 



56. The method of claim 1, wherein the step of generating of the three 
5 dimensional image of the structure includes a Mercator Virtual Pathology. 

57. The method of claim 1 , wherein the step of generating of the three 
dimensional image of the structure includes a Cylindrical Virtual Pathology. 

10 58. The method of claim 1, wherein the step of generating of the three 
dimensional image of the structure includes a Planar Virtual Pathology. 

59. The method of claim 1 , wherein the step of generating of the three 
dimensional image of the structure includes an Axial Planar Virtual Pathology. 

15 

60. The method of claim 5, wherein the step of generating of the three 
dimensional image of the structure includes a panoramic Virtual Endoscopy. 

6 1 . The method of claim 5, wherein the step of generating of the three 
20 dimensional image of the structure includes a Mercator Virtual Pathology. 

62. The method of claim 5, wherein the step of generating of the three 
dimensional image of the structure includes a Cylindrical Virtual Pathology. 

25 63. The method of claim 5, wherein the step of generating of the three 
dimensional image of the structure includes a Planar Virtual Pathology. 

64. The method of claim 5, wherein the step of generating of the three 
dimensional image of the structure includes an Axial Planar Virtual Pathology. 
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65. The system of claim 20, wherein generating of the three dimensional image 
of the structure by the render means includes a panoramic Virtual Endoscopy. 

66. The system of claim 20, wherein generating of the three dimensional image 
5 of the structure by the render means includes a Mercator Virtual Pathology. 

67. The system of claim 20, wherein generating of the three dimensional image 
of the structure by the render means includes a Cylindrical Virtual Pathology. 

10 68. The system of claim 20, wherein generating of the three dimensional image 
of the structure by the render means includes a Planar Virtual Pathology. 

69. The system of claim 20, wherein generating of the three dimensional image 
of the structure by the render means includes an Axial Planar Virtual Pathology. 

15 

70. The system of claim 24, wherein generating of the three dimensional image 
of the structure by the render means includes a panoramic Virtual Endoscopy. 

71 . The system of claim 24, wherein generating of the three dimensional image 
20 of the structure by the render means includes a Mercator Virtual Pathology. 

72. The system of claim 24, wherein generating of the three dimensional image 
of the structure by the render means includes a Cylindrical Virtual Pathology. 

25 73 . The system of claim 24, wherein generating of the three dimensional image 
of the structure by the render means includes a Planar Virtual Pathology. 

74. The system of claim 24, wherein generating of the three dimensional image 
of the structure by the render means includes an Axial Planar Virtual Pathology. 

30 
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75. The one or more program storage devices of claim 40, wherein the step of 
generating of the three dimensional image of the structure includes a panoramic 
Virtual Endoscopy. 

5 76. ' The one or more program storage devices of claim 40, wherein the step of 
generating of the three dimensional image of the structure includes a Mercator 
Virtual Pathology. 

77. The one or more program storage devices of claim 40, wherein the step of 
10 generating of the three dimensional image of the structure includes a Cylindrical 

Virtual Pathology. 

78. The one or more program storage devices of claim 40, wherein the step of 
generating of the three dimensional image of the structure includes a Planar Virtual 

15 Pathology. 

79. The one or more program storage devices of claim 40, wherein the step of 
generating of the three dimensional image of the structure includes an Axial Planar 
Virtual Pathology. 

20 

80. The one or more program storage devices of claim 54, wherein the step of 
generating of at least the two three dimensional images of the structure includes a 
panoramic Virtual Endoscopy. 

25 81 . The one or more program storage devices of claim 54, wherein the step of 
generating of at least the two three dimensional images of the structure includes a 
Mercator Virtual Pathology. 

82. The one or more program storage devices of claim 54, wherein the step of 
30 generating of at least the two three dimensional images of the structure includes a 
Cylindrical Virtual Pathology. 
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83 . The one or more program storage devices of claim 54, wherein the step of 
generating of at least the two three dimensional images of the structure includes a 
Planar Virtual Pathology. 

84. The one or more program storage devices of claim 54, wherein the step of 
generating of at least the two three dimensional images of the structure includes an 
Axial Planar Virtual Pathology. 
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